[LI-HOTFIX] Improve unofficial client log message and add more default software names #202
Conversation
…t software names TICKET = N/A EXIT_CRITERIA = N/A Add client's software name and version to log message. Add software names for Admin client to default software names list config.
| @@ -1696,11 +1696,13 @@ class KafkaApis(val requestChannel: RequestChannel, | |||
|
|
|||
| if (config.unofficialClientLoggingEnable) { | |||
| // Check if the last part of clientSoftwareName (after commitId) is an unexpected software name | |||
| val softwareName = apiVersionRequest.data.clientSoftwareName().split("-").last | |||
| val softwareNameAndCommit = apiVersionRequest.data.clientSoftwareName() | |||
There was a problem hiding this comment.
Seems there are couple of our internal services that uses unofficial clients (open source clients), we might want to add those internal service to our expected names as well. But not all of them can be uniquely identified from the client ID. Here's the summary from logging in ei-ltx1 tango cluster:
- menagerie: uses open source admin client, has string menagerie in principal
2021/09/02 18:00:15.966 WARN [KafkaApis] [data-plane-kafka-request-handler-118] [kafka-server] [] [KafkaApi-10309] received ApiVersionsRequest from user with unofficial client type. clientId clientAddress principal = adminclient-39 /2a04:f547:24:4401:0:0:0:585e User:menagerie:None:ei-ltx1
- CC -> uses open source producer/consumer which has string CruiseControl in client id; uses open source admin client has string cruise-control in principal
2021/09/02 17:24:36.343 WARN [KafkaApis] [data-plane-kafka-request-handler-2] [kafka-server] [] [KafkaApi-10309] received ApiVersionsRequest from user with unofficial client type. clientId clientAddress principal = CruiseControlMetricsReporter /2a04:f547:24:470d:0:0:0:736c User:kafka:kafka.tango:ei-ltx1
2021/09/02 18:24:22.625 WARN [KafkaApis] [data-plane-kafka-request-handler-118] [kafka-server] [] [KafkaApi-10309] received ApiVersionsRequest from user with unofficial client type. clientId clientAddress principal = adminclient-1 /2a04:f547:24:440c:0:0:0:5b1d User:likafka-cruise-control:None:ei-ltx1
- Conductor-service: uses open source admin client, has string conductor-service in principal
handler-20] [kafka-server] [] [KafkaApi-10309] received ApiVersionsRequest from user with unofficial client type. clientId clientAddress principal = adminclient-2 /2a04:f547:4a:674:0:0:0:dd1d User:conductor-service:conductor-service.cert1:ei4
- Kafka-rest -> didn't seem to find any open source clients usage in kafka-rest, need the software name logging in this patch to see where the unofficial client is from; has string tracking-rest in principal
2021/09/02 18:10:13.044 WARN [KafkaApis] [data-plane-kafka-request-handler-99] [kafka-server] [] [KafkaApi-10309] received ApiVersionsRequest from user with unofficial client type. clientId clientAddress principal = tracking-rest-restObserver.producer /2a04:f547:24:4301:0:0:0:3860 User:tracking-rest:tango:ei-ltx1
2021/09/02 18:10:18.601 WARN [KafkaApis] [data-plane-kafka-request-handler-18] [kafka-server] [] [KafkaApi-10309] received ApiVersionsRequest from user with unofficial client type. clientId clientAddress principal = tracking-rest-kafkaTracking.trackingProducer /2a04:f547:24:4511:0:0:0:646b User:tracking-rest:tango:ei-ltx1
So I'm thinking besides the default expected client names from lkc mp (e.g. AvroKafkaProducerFactory etc), we should also add another set of expected service names {cruise-control, tracking-rest, conductor-service, menagerie} and expected client ids (CruiseControl) and not logging for clients from these services?
@radai-rosenblatt @efeg @xiowu0 any thoughts?
There was a problem hiding this comment.
all these cases just need to be fixed, instead of allowed.
TICKET = N/A
EXIT_CRITERIA = N/A
Add client's software name and version to log message.
Add software names for Admin client to default software names list config.
No testing done.
Committer Checklist (excluded from commit message)